Document Cloud RDI pipeline capacity - #4085
Open
ZdravkoDonev-redis wants to merge 11 commits into
Open
ZdravkoDonev-redis wants to merge 11 commits into
ZdravkoDonev-redis wants to merge 11 commits into
Conversation
Contributor
Contributor
🧠 Redis MemoryFound 11 related items from repository history:
Memory updated at 3d05714 |
ZdravkoDonev-redis
force-pushed
the
cloud-rdi-processor-scaling-docs
branch
from
September 25, 2026 13:32
a8f33df to
2819c3f
Compare
ZdravkoDonev-redis
force-pushed
the
cloud-rdi-processor-scaling-docs
branch
from
September 25, 2026 18:15
2819c3f to
1cd9724
Compare
ZdravkoDonev-redis
changed the title
ZdravkoDonev-redis
marked this pull request as ready for review
September 25, 2026 19:22
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Before and after
Before this change, Cloud RDI documentation did not explain whether processor capacity was reactive autoscaling, how to set a replica count, or how to confirm it beyond billing. After this change, a user can set a Flink processor to three TaskManagers and use the Dashboard value 3 / 3 replicas to confirm that three are running and three are configured.
Validation
Replaces #4081, which GitHub closed when its invalid slash-containing head branch was renamed.
Note
Low Risk
Documentation-only updates to Cloud RDI operate guides; no product code or runtime behavior changes.
Overview
Adds Scale a Cloud RDI pipeline (
scale-pipeline.md) with aliases from older scale-processor URLs, covering how to raise capacity at the collector (Debezium tuning), RDI database throughput tiers, and Flink TaskManagers viaadvanced.resources.taskManager.replicas(1–10), plus when to scale and how to verify replicas on the Dashboard (running / configured) or pipeline-status API—not billing or Metrics.The RDI Cloud FAQ gains Capacity and network planning (10 sources / 10 TaskManagers,
/22vs/21CIDR, non-expandable workspace CIDR) and Processor scaling (explicitly no autoscaling, tuning steps, console/API confirmation).Data Integration landing and View and edit → Settings now link to the scaling guide for processor capacity.
Reviewed by Cursor Bugbot for commit 3d05714. Bugbot is set up for automated code reviews on this repo. Configure here.